System testing checks whether the system under test operates as intended or not. When the system under test does not operate as intended, yielding unexpected results, this is due to defects in the system or the system's software. The correction of these defects, and the introduction of new features or the enhancement of existing features, may introduce new defects into the system under test. Repeated testing improves the likelihood of catching defects soon after their introduction.
In general, there are two testing approaches with mutually exclusive advantages, namely, manual testing and automated testing. The strength of manual testing, especially in complex software environments, is that the software is used from the customer's point of view and with a customer-like behavior thereby executing an arbitrary sequence of activities and inputs. Due to manual testing the state of the system under test is non-deterministic, e.g. due to user errors. Thus, the system under test might get into a system state that has not yet been considered. In addition, long term effects might be detected across the complete set of system features and belonging to totally different system blocks.
In contrast thereto, automated testing shows other advantages. Automated testing is less resource intensive than manual testing because no human test person is needed. In addition, occurring failures are deterministically reproducible because of the well-defined run of the test routine. Finally, automated testing is much faster, i.e., a higher coverage of testing is reached in a certain period of time.
Known solutions and tools essentially are focused on automated test execution. By that, these solutions provide the advantages of automated testing as described above but do negate the benefits of manual test execution. Hence, once manual test scenarios get automated, most of the mentioned advantages of manual testing are lost because the user-specific randomness is eliminated by using a deterministic test environment. In addition, most automated tests do not focus on long-term effects and do not reflect stochastical aspects of human user activities.
Therefore, it would be advantageous to have a mechanism for system testing which combines the advantages of manual and automated testing in order to improve the quality and error detection of automated system testing.